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Abstract 


Higher order thinking skill (HOTS) is one of the students’ abilities that should be developed through 
teaching and learning. Teachers’ knowledge about HOTS and its teaching and learning tactics is a key to 
successful education. The purpose of this research is to describe teachers’ knowledge about higher order 
thinking skills (HOTS). The research involves qualitative study with the phenomenological approach. 
The research participants are 27 mathematics teachers from state and private junior high schools across 
7 provinces in Indonesia. The researcher collected data with a test followed by focus group discussion 
(FGD) and interviews. The analysis of data involved Bogdan & Biklen model and descriptive statistics 
for data from the test. The analysis of FGD, and test data intends to get information on 6 sub-themes; 
teachers’ knowledge about HOTS, importance of HOTS, teaching about HOTS to students, improving 
students’ HOTS, measuring and assessing HOTS, and teachers’ ability for solving HOTS-based problems. 
The results indicate that teachers’ knowledge about HOTS, their ability to improve students’ HOTS, solve 
HOTS-based problems, and measure students' HOTS is still low. There are facts, however, that teachers 
already understand the importance of HOTS and teaching it by using various innovative learning models. 
Keywords. HOTS, measurement and assessment, teachers’ knowledge, teaching and learning 


Introduction 


Education is one of the important aspects that can affect national vision. Success in edu- 
cational implementation is a key to a better future. The latter cannot be achieved without the 
contribution of various stakeholders. The government, as a policymaker, has the main role and 
responsibility to pave the way to success, by pledging education for all throughout the nation, 
for example, education for free policy in developing countries (Quamruzzaman, Rodriguez, 
Heymann, Kaufman, & Nandi, 2014). Aside from the government’s role, teachers also play an 
overwhelming role because they put in practice all educational policies and regulations. 

Education quality is not only determined by the plan and the development of education 
but also the quality of its implementation. It also requires supervision of different partners, such 
as education experts, practitioners, and stakeholders. All of them must synergize to achieve 
educational success, this means that it not only focuses on the improvement of evaluation, 
updated curriculum, and new regulations, but also on the development of teaching profession- 
alism (Purnomo, 2017). Some studies revealed that teachers are already a great contribution 
to improving education quality in many countries (Gil-Flores, Rodriguez-Santero, & Torres- 
Gordillo, 2017; Hu, Fan, Yang, & Neitzel, 2017; Stylianides, 2007). Therefore, the quality of a 
teacher to implement educational policy affects the quality of education. 

The efforts to improve the quality of education cannot be separated from the demands of 
21st century competitiveness which is complex and challenging. There are three main frame- 
works of 21st-century skills: 1) learning and innovation skills, 2) life and career skills, and 3) 
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information, media, and technology skills (Scott, 2017). Learning and innovation skills consist 
of communication, collaboration, critical thinking, and creativity skills (4CS). Life and career 
skills consist of flexibility and adaptability, initiative and self-direction, social and cross-cultur- 
al skills, productivity and accountability, leadership, and responsibility. Furthermore, informa- 
tion, media, and technology skills consist of information and media literacy. In addition, Bialik 
(2015) mentioned four key issues in the 21st-century education implementation framework: 
Knowledge, skill, character, and metacognition. 

Related to the skill, Bialik (2015) and Scott (2017) mentioned that it is relevant to the 
4CS competence, which includes creativity, critical thinking, communication, and collabora- 
tion. Scott (2017) and Bialik (2015) stated that the 21st-century skills can be grouped into two 
main components: abstract skills related to thinking skills (creative thinking and critical think- 
ing), and concrete skills (communication and collaboration). Moreover, creative thinking skills 
and critical thinking skills are included in higher-order thinking skill (HOTS) (Miri, David, & 
Uni, 2007; Moseley, Baumfield, Elliott, Gregson, Higgins, & Newton, 2005). 

HOTS is one of the important components for an individual to be able to solve new prob- 
lems in the 21st century (Brookhart, 2010; Moseley et al., 2005; Thompson, 2008). HOTS also 
plays an important role in applying, connecting, or manipulating the prior knowledge in order 
to effectively solve new problems (Thomas & Thorne, 2009). In the revised Bloom’s taxonomy, 
HOTS is defined as an incision among the three top levels of ability in the cognitive dimension 
(analyzing, evaluating, creating), and 3 levels of knowledge dimension (conceptual, proce- 
dural, metacognitive) (Anderson & Krathwohl, 2001, Thompson, 2008). Therefore, HOTS is 
measured using tasks, including analyzing, evaluating, and creating conceptual and procedural 
knowledge, or metacognition. This means that familiarizing students with HOTS activity is 
important to help them get ready for solving new issues, acclimatizing themselves in a new 
atmosphere, and making decisions about a particular problem. 

In the implementation of learning, HOTS cannot be directly taught to students. Students 
should be trained about HOTS, as a skill, through learning activities that support its devel- 
opment. Active learning and student-centered learning are activities for training about HOTS 
(Akyol & Garrison, 2011; Limbach & Waugh, 2010). The active learning and student-centered 
learning such as problem-based learning (PBL) (Mokhtar, Tarmizi, Job, & Nawawi, 2013), 
project-based learning (Pj]BL) (Vidergor & Krupnik-Gottlieb, 2015), discovery learning, and 
inquiry-based learning (Orlich, Harder, Callahan, Trevisan, & Brown, 2010), or other learning 
models using contextual problems are some examples of strategies for training students about 
HOTS. In addition, Protheroe (Goethals, 2013) and Miri, David, and Uri (2007) mentioned that 
group discussion and solving complex and interdisciplinary problems in the learning process 
are important activities to train students’ HOTS. 

The outcome from students’ HOTS development can be achieved by the active role 
of teachers in planning, implementing, and evaluating HOTS-oriented learning. To be able to 
plan HOTS-oriented learning, teachers need knowledge of ways, strategies, methods to train 
students about HOTS (Bartell, 2012). Some research results, however, indicate that mathemat- 
ics teachers face some difficulties, particularly when training their students about HOTS. The 
obstacles they encounter including difficulty to develop HOTS-based problems and find suited 
learning tools (Jailani & Retnawati, 2016; Retnawati, Munadi, Arlinwibowo, Wulandari, & 
Sulistyaningsih, 2017; Thompson, 2008), there are also difficulties to evaluate students’ HOTS 
(Retnawati, Hadi, & Nugraha, 2016). Various obstacles also implicate the low-level of students’ 
HOTS, they mark difficulties when solving mathematics problems in the national exam with 
contextual problem and narration (Retnawati, Kartowagiran, Arlinwibowo, & Sulistyaningsih, 
2017). 
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Research Problem 


Based on the description, it is still needed to conduct the assessment of teachers’ knowl- 
edge about HOTS and its learning strategies. Throughout this research, it is expected to get a 
broad picture of teachers’ knowledge about HOTS and its learning strategy. This information 
assists in planning programs that can increase teachers’ knowledge about HOTS and its learn- 
ing strategy. Therefore, this study intends to describe teachers’ knowledge about HOTS and its 
teaching strategy in mathematics classroom. 


Researcher's Role 


In this research, the only relationship between the researcher and participants is to col- 
lect and explore data with test, focus group discussion (FGD), and interviews resulted from 
teacher knowledge about HOTS. The next step was to analyze the data for having advanced 
understanding. The researcher did not undertake any treatment upon the participants in terms 
of the teacher’s knowledge and mathematics teaching and learning strategies implemented by 
the teachers in the classroom. 


Research Methodology 
Research Type 


This research is a qualitative study in the phenomenological approach. This research 
aims to explore teachers’ knowledge of HOTS and their teaching strategy in mathematics class- 
room. 


Research Participants 


The research participants were 27 mathematics teachers (14 female against 11 male; T1- 
T27) from 13 private Junior High Schools and 14 state Junior High Schools from 7 provinces 
in Indonesia. They were randomly selected. All teachers have more than 10 years of experience 
teaching Mathematics in Junior High School. They haven’t participated in training for HOTS 
and its teaching and learning strategy yet. 


Instrument and Procedure 


The teachers passed a test, involved in Focus Group Discussion (FGD) and in-depth in- 
terviews so that the researcher could get detailed information. The test consists of 2 constructed 
response items that adapted from PISA released items. Two mathematics education experts 
had validated the items. FGD and interviews topics consist of six sub-themes: 1) Teachers’ 
knowledge on HOTS, 2) importance of HOTS, 3) learning strategy in implementing HOTS, 4) 
improving students’ HOTS, 5) measuring and assessing HOTS, and 6) teachers’ ability to solve 
HOTS problems. The test was also administered to measure teachers’ HOTS by giving them 
HOTS problems in Figure 1. 


Data Analysis 
Data from FGD and interviews were analyzed and presented in a table to be classified 
into sub-themes. Analysis of data from the FGD involved Bogdan and Biklen model (1982) in 


order to know the relationship among sub-themes. On the other hand, the analysis of teachers’ 
answers about solving HOTS problems involved descriptive analysis. 
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Ethical Consideration 


At the beginning of the FGD and interviews, the researcher informed the participants 
that the study would only capture their knowledge about HOTS and its teaching and learning 
strategy. The FGD and interviews were naturally conducted to obtain credible information. The 
teachers’ names were kept and for the purpose of the research, their identities were coded to 
ensure their answers have no effect on their professions as teachers. 





1. Magazine Subscription 
Geo-Picture magazine monthly subscription: 























Geo-Picture Magazine Category Price List 

Geo-Picture Indonesia Rp50.000,00 
Geo-Picture Traveler Rp45.000,00 
Geo-Picture for Kids Rp20.000,00 





he magazine also offers annual subscription packages. If you subscribe at least two magazines in a year, you will get 
the discounts which are listed below: 


























Geo-Picture Magazine Category pace ltet 
Geo-Picture Indonesia Geo-Picture Traveler Geo-Picture for Kids 
Rp600.000,00 
Rp600.000,00 
Rp444.000,00 
Rp804.000,00 








f Rudi wants to subscribe the magazines, which one is the cheapest package that should be choosen by Rudi? 
Explain! 


2. Dice game 

Two players threw a dice. From the dice number that appears, the larger dice number is reduced by the smaller dice 
number. If the difference is 0, 1, or 2 then player A gets 1 point. However, if the difference is 3, 4, or 5 then player B 
gets 1 point. The game ends after 12 dice and the player with the most points is the winner. 

Is the game fair? If the game were unfair, how would the rules be changed to make the game fair? 











Figure 1: The test to measure teachers’ ability in solving HOTS problems. 
Research Results 


Result gives information about teacher’s perceptions towards mathematics learning to 
improve HOTS. The teachers’ perceptions are classified into teachers’ knowledge on HOTS, its 
importance, its implementation learning strategy, its improvement in students, its measurement 
and assessment, and teachers’ ability to solve HOTS-based problems. The researcher explained 
every sub-theme here below. 


Teachers’ Knowledge about HOTS 


The results out of data analysis and reduction of teachers’ knowledge about HOTS show 
that their knowledge about HOTS is still low. Some explanations given by teachers are general. 
In fact, there are many teachers who are still confused about distinguishing between HOTS and 
its strategies or learning methods. The same results can be seen in Table 1. The findings indicate 
that some teachers have explained HOTS using the three top levels of the revised Bloom's tax- 
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onomy (analyzing, evaluating, and creating) while others use critical thinking skill, creativity, 
problem-solving, logic, reflective, and metacognitive. Based on the FGD result, new teachers 
are able to understand conceptual knowledge, but cannot explain the operational knowledge 
properly. Based on teachers’ responses, not all teachers understand HOTS well. Teachers still 
cannot differentiate among abilities, skills, learning methods or learning activities. Although the 
term HOTS always appears in teacher training activities and socialization of Curriculum 2013, 
teachers cannot understand its definition. 
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Table 1. Teachers’ knowledge about HOTS definition. 


Definition of HOTS according to mathematics teachers Verification result 





Not all teachers understand 
HOTS well. Teachers are still 
unable to distinguish HOTS as 
an ability, skill, learning strategy, 
learning method, or learning 
process. 


Learning stage that requires mastery of the correct concept 

Thinking skill that is more than just memorizing and reading 

High cognitive process, such as analysing, evaluating, and creating 

Thinking ability through understanding, observing, exploring, and inferring data 
Critical thinking skill, creativity, logic, problem-solving, and metacognitive 
Learning process which includes knowledge, skill, and analysis 

High-level problem-solving ability 

Problem analysis skills 


Learning methods which are not just for solving a problem, but understanding it at 
a high level 


Ability to solve problems with various ways 





Teachers’ Knowledge about the Importance of HOTS 


Table 2 contains the findings out of data analysis and reduction for teachers' knowledge 
about HOTS. Teachers’ responses to the second sub-theme prove that most of the teachers said 
that HOTS is important because of its advantages in solving various and complex problems. 
Therefore, HOTS-oriented learning is very important in learning process as far as it helps solve 
daily life problems. 


Table 2. Teachers’ knowledge about the importance of HOTS. 





The importance of HOTS Verification result 


In learning mathematics, students’ task is not mastering the concept but also engaging 
problem-solving skills 





Training both students’ thinking and problem-solving skills 
Making a coherent learning process, such as analysing problem and evaluating 


Being able to think and carefully solve very difficult problems Kicelteadions have eaibed 


Training students in critical thinking 


Not separating HOTS from the cognitive elements of critical thinking skills, creativity, 
problem-solving, logic, reflective, and metacognitive 


Needing knowledge, skill, and analysis skill in solving problem that might be faced in 
daily life 


HOTS is required to solve problem 
Reaching out routine and unexpected problems (non- routine problem) 
Studying easily by using HOTS 


the importance of HOTS 
for students, i.e. improving 
students’ skills in solving 
daily life problems. 
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There are other responses which show the importance of HOTS, it makes students easier 
to learn. Based on their responses, some teachers still argue about the importance of HOTS. 
They stated that HOTS is used to train children to think critically (T5). By teachers’ responses, 
it is worthy to conclude that teachers already know the importance of HOTS. The majority of 
respondents are aware of the importance of HOTS. 


Teachers’ Knowledge about Implementing HOTS in Learning Process 
For the third sub-theme, teachers were asked to explain the strategies for teaching HOTS. 
In the same sub-theme, teachers are expected to describe learning model that can be suited to 


teaching HOTS in mathematics topics. The results can be seen in Table 3. 


Table 3. Teachers’ knowledge about implementing HOTS in learning process. 


Ways of implementing HOTS in learning Verification result 





Learning atmosphere should be fun, mastering of basic concepts must be strong, using small Teachers know that 
groups, using HOTS questions as a habit, developing perseverance and curious attitude teaching HOTS can 
be done by using vari- 


Using problem-solving approach and giving analysis-related questions ous learning models. 


Teaching students to analyze, evaluate, and create systematically Teachers are able to 

Training students to use complex questions and articulate problems, and asking them to find Mention a number of 

an answer by following a particular procedure leaming models for 
teaching HOTS. 


Providing problems related to their daily problems 

Applying problem-solving approach 

Giving students problems or HOTS-based questions 

Understanding, identifying, determining formulas, and solving problems 
Giving PISA questions to students 


Applying problem- based learning, project-based learning, active learning, inquiry learning, 
and cooperative learning 





Results show that teachers’ knowledge on teaching HOTS has a positive impact. The 
teachers believe that training students’ HOTS can be done through various learning models. 
The results also indicate that teachers already know that teaching HOTS to students can be 
carried out by learning activities containing problem-solving. In addition, teachers' responses 
proved their knowledge about problem-based learning, project-based learning, active learning, 
inquiry learning, and cooperative learning. 


Teachers’ Knowledge about Improving Students’ HOTS 


The results about teachers’ knowledge on improving students’ HOTS can be seen in 
Table 4. In this sub-theme, teachers were asked to explain learning activities which can im- 
prove students’ HOTS, the results indicate that they are still unable to explain how to improve 
students' HOTS, either conceptually or operationally. 

Teachers’ answers show their misunderstanding about the operational implementation of 
learning activities to train about HOTS. Most of the answers given by teachers are still norma- 
tive, such as “asking students to think critically”, “Continuously training them about thinking 
skills”, and “training them by asking questions, creating group discussions, giving analysis 
(C4) or evaluation (C5) related questions”’. 
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Table 4. Teachers’ knowledge about improving students’ HOTS. 221 
Mechanisms to improve students’ HOTS Verification result 
Learning habit and mentorship Most teachers still cannot 


explain how to improve 
students' HOTS, either 
conceptually or operation- 
ally. 


Engaging practice by asking questions, creating discussion groups, giving C4 or C5 
questions 


Making games followed by increasing the level of thinking 

Training by providing questions that contain high-level thinking process 
Applying problem-based learning model 

Designing methods, techniques, or approaches which can improve HOTS 
Giving students problems ordered from low to high level 

Providing students with problems from easy to difficult level 

Assigning students to read more HOTS literature 


Reading a lot of reference books and exchanging knowledge, methods, or knowledge 
with friends 


Asking students to think critically 





Teachers’ Knowledge about Measuring and Assessing HOTS 
The results from data analysis and reduction are summarized in Table 5. Their responses 
indicate that most teachers can measure and assess HOTS, by describing, confirming based on 


observation and presentation, and scoring. 


Table 5. Teachers’ knowledge about measuring and assessing HOTS. 





How to measure and assess HOTS Verification result 


Conducting interview with some students on the difficulty of HOTS problems Most teachers already have idea 
about appropriate instruments to 
assess HOTS, such as essays, 
observation of problem-solving 
process, confirmation relying on 
observation and presentation, and 
Analysing students’ answers especially in the process of completing the answers _ scoring system. 


Measuring students’ understanding on solving problems, constructing and find- 
ing solutions, and evaluating the outcomes 


Observing the effort of students in thinking and solving their problems, by 
themselves 


Using written assessment and observation 


Conducting assessment of the process and final evaluation. Developing an 
instrument measuring high-level skills 


Using essays, assessing the process of finding solutions and stating final solu- 
tion, interviewing students or assessing their presentations 


Giving continuous problems and observing the improvement 
Giving essays rooted on students' daily life problems 
Using assessment sheet, essay test, and open-ended problems 





The results make it clear that teachers already know about different instruments suited to 
measuring HOTS, such as essays with contextual problems. Other responses explain the tech- 
niques for assessing procedures for not appraising the outcome only. These results indicate that 
teachers already know the instruments used to conceptually measure HOTS. These responses, 
however, only explain the assessment process regarding problem solving steps, none of them 
clarifies how to measure or assess HOTS with Bloom’s taxonomy. 
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Teachers’ Ability for Answering to the HOTS-Based Questions 


In this study, the researcher asked teacher to find answers to some problems in order 
to find out teachers’ HOTS. Examples of teachers’ answers in solving the first HOTS problem 
are listed below. 

“The cheapest package is the fourth one because if the normal price per annual 
subscription is calculated, the first package price drops by Rp540.000,00, the second by 
Rp240.000,00, and the third Rp336.000,00, and the fourth Rp576.000,00” (T1) 

“The packages that should be chosen are Geo-Picture Indonesia and Geo- 
Picture Traveler which is only for Rp600.000,00 because the subscription price of each 
magazine is cheaper than Geo-Picture for Kids” (T2) 

“T can choose Geo-Picture Indonesia, Geo-Picture Traveler, and Geo-Picture 
for Kids for Rp804.000,00 because they have the biggest discount” (T3) 

“IT can choose 3 books from the first package because if the price is accumu- 
lated, it has more discount” (T4) 

“TI can choose Geo-Picture Indonesia and Geo-Picture Traveler package be- 
cause the discount is bigger than other packages” (Teacher 5) 

From the five answers, the right answer is the answer from Teacher 1, He systemati- 
cally solved the problem of “Magazine Subscription” and identified important information by 
analyzing, investigating, solving problems (creating), evaluating and drawing conclusions. The 
way he solved the problem is presented in Figure 2. 


Wf Rudi wants to subscribe the magaziec, whic hh one is the ceapest package that should be choosen by Rudi? 








Explaint a+b « 1- 140.000 - £00 
Geo - picture Indonesia = & = F40-CD° 
” Traveler +: o arte 840.000- 


= for Kids + = 240-000 


a = 12% s0.coms 600-655 
b = (2 so .c20 - SYO 09° 


= (2X20 OOO EE R4OCOO 


J3jeor => 





== atrvor+ e =a A. 3B0 .90O0 
—- 8 O0Y¥.0cCOo 
= 5 76.000 
The ceapest paceage is) & + 4+ 2 of 
ey rae ja. Goo-pichre Troveler + 
- fwe indonesia + 
Geo - pre 


Creo - picture tc?" 
Figure 2: Answer by teacher 1. 


Figure 2 is an example of the correct answer. Teacher | solved the problem by separating 
the magazines, determined the price of each magazine within one year, and then the discount for 
each package. After that, he took the decision by considering the biggest discount. Therefore, 
according to Teacher 1, the cheapest package that should be chosen by Rudi is the fourth one. 
Teacher 3 and Teacher 4 actually have the correct answer, but their answers are not built on 
problem-solving process. Figure 3 contains the work by teacher 3. 
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If Rudi wants to subscribe the magazie, which one is the ceapest package that should be choosen by Rudi? 


Explain! 
“picture Indonesia. WH Geo -fricture Fer and 
eo- pichne for Kide Be ¥0#.000,00 cause 76 


much al Cound: | 7 
Wop - » ty Ge. buy et f 


Figure 3: Answer by teacher 3. 


Figure 3 is an example of the correct answer but not complete with clear and detailed 
completion process. It can be seen in Figure 3 that Teacher 3 chose the fourth package because 
of its discount. The discount referred to by Teacher 2, however, is not included in the process 
of drawing his conclusion. The way of understanding of teacher 2 is still partial in solving 
HOTS-based problems. In contrast, Teacher 2 and 5 gave different answers from three other 
teachers. Teacher 2 selected the second package because he compared prices of all magazines, 
whereas Teacher 5 preferred the first package because of its price. Figure 4 illustrates the work 
of Teacher 2 on the problem. 


If Rudi wants to subscribe the magazie, which one is the ceapest package that should be choosen by Rudi? 
Explain! 


Audi ohauld chaase Geo-pretire Indanesia and 


Gea-picture Traveler 95 much os 600.005 ._ 
Cause the unit cost oF Gen picture \ndonesa 
a Geo -picture Treveler are the most 

an 


expensive. 


Figure 4: Answer by teacher 2. 


Figure 4 is an example of the wrong answer. The mistake of teacher 2 lies within his pro- 
cess of drawing conclusion, the process is not accompanied with a clear completion. Teacher 2 
selected the first package (Geo-Picture Indonesia and Geo-Picture Traveler) because the price 
of each magazine in the package is the most expensive comparing to each price of the Geo- 
Picture Traveler. The reason was given by Teacher 2, however, is not appropriate to answer 
the question. There are two possibilities that cause teachers to make wrong answers. First, the 
teacher does not understand the question. Second, the teacher does not understand the discount 
concept. Therefore, it can be concluded that Teacher 2 skills to solve the HOTS problem are 
still low. 

For the second HOTS-based problem, the teachers’ answers vary. The sample of them 
is listed below. 
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“In my opinion, the game is fair” (T7, T9, without reason) 

“A is as probable as B, 2” (T15, without mentioning fair or not fair) 

“The game is not fair because the probability of A is different from that of B.” (T20, T24) 

The answers of T7, T9, and T15 are wrong, and the answers of T20 and T24 are partially 
correct. The correct answer is from T25, Figure 5. 


ae) he SRS a 





A and &@ dont get 
2.1 the number in acce 


oF A «Ss more than B, 
A get 1 point 





Agference O.1,2 7 A oes 1 RIN 3- *€ the number in dice 
dxtterence 3,4,5 > & 9etS 1 Poin of B (S more thon A, 
gCAa)= 7 $ B get 1 point 
1. ¥. 


RCB>)= 20°? 


Probab (rey A to geek point 
more than g. 
The game is not fair. 


Figure 5: Answer by teacher 25. 


Many teachers have partially made correct answers. They wrote that the game is not 
fair, but they didn’t give the reason behind or propose the new rule to turn the game fair. For 
example, the T22 answer is illustrated in Figure 6. In this answer, the teacher wrote the analysis 
to make the new rule but did not write the rule. From the answers of teachers to the test, the 
teachers’ skills to solve HOTS problems in mathematics are still low. 






me is Unfat 

Phe Shovid be ChAuges * 

A. Che difRerewe of 2 4°¢05,2 4 
Chetavse 2 Can appéare 8 Ces, K 

' aq abheare 10 G'mes ). 


B. ésc OF R era og 2 brces TL30 


Chetayse O Can dahPewe ¢ brome 





3 M © ken cs ; 
Lf + 4 Crone 
> —_/—— > Creues 9 


Figure 6: Answer by teacher 22. 
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Discussion 


Curriculum 2013 is the latest curriculum that has been implemented in Indonesia since 
2013. One of the components of the Curriculum is higher-order thinking skill (HOTS). Ac- 
cording to the revised Bloom’s taxonomy (Anderson & Krathwohl, 2001), HOTS is an incision 
between the three top components of cognitive process dimension (analysis, evaluation, and, 
creation) with the three top components of knowledge dimension (conceptual, procedural, and 
metacognitive). Based on the definition, the possibility of teachers to understand the whole con- 
cept of HOTS is still lacking. Moreover, teachers will have hugely difficult moments to apply 
HOTS in learning process, if not accompanied by socialization and training from the govern- 
ment on a regular basis. 

The findings of this study indicate that teachers still misunderstand HOTS. Some teach- 
ers assume that HOTS is the learning phase (Teacher 1 and Teacher 7). Another teacher also as- 
sumes that HOTS is a method of learning (Teacher 10). Considering the fact that some teachers 
still partially understand HOTS, socialization and training are still needed in order to introduce 
HOTS to mathematics teachers. The quality of socialization and training is important so that 
teachers can get more understanding ability and skills about HOTS through these activities. 
Retnawati (2015) also stated that based on the qualitative study, teachers' trainings and Cur- 
riculum 2013 socialization are still insufficient. Some issues are found in teacher training and 
socialization, such as multiple interpretations in terms of training and socialization themes and 
time limitations that lead to incomplete delivery of the materials. These issues need to be con- 
sidered for further training and socialization. 

HOTS is one of the important things that become the orientation key of educational 
policy implementation. There are two main reasons that students have to perform successfully 
in schools and they should make a positive contribution to the society (Conklin, 2012). There- 
fore, HOTS is very important to be applied in learning process so that students get ready to 
contribute to the society. In learning mathematics, HOTS is one of the determinant factors of 
student success. The complexity of materials and problems in mathematics also requires edu- 
cators; teachers and students; to have higher-order thinking skills. Both students and teachers 
should realize the importance of HOTS for students. 

The outbreak of HOTS issues in the implementation of learning in Indonesia requires 
empirical changes. Ahmad (2014) revealed that there are two teacher perceptions about the edu- 
cational change, positive perceptions and negative perceptions. Positive perceptions cultivate a 
desire for change and innovation, whereas negative perceptions indicate teachers’ unprepared- 
ness to make changes. Both teachers and student are related to the urgency of HOTS. A study 
conducted by Avargil, Herscovitz, and Dori (2012) found that students also support teacher 
professional development, and so do the teachers. It can be seen from one teacher’s response 
that HOTS is important “because we encounter some problems that need knowledge, skills, 
and analysis so that we can fix our daily life problems”’. In the response, teachers use the word 
“us” instead of “students” or “them”, which means that HOTS is not only needed by students 
but also by their teachers. 

The results show that teachers have realized the importance of HOTS. Teachers’ aware- 
ness demonstrates that they are ready to make changes or improvements during learning pro- 
cess. Although some previous research (Jailani & Retnawati, 2016; Retnawati, 2015; Retnawati 
et al.; 2016; Retnawati, et al., 2017) demonstrated a number of teacher difficulties in imple- 
menting learning or assessment model that fits the demands of the Curriculum, the results of 
this study indicate that teachers believe the importance of implementing HOTS in the learning 
process. This belief will foster the spirit of teachers in making innovation and change which in 
line with the positive perceptions of teachers towards the curriculum changes and also foster 
teachers’ desire to innovate in order to support the implementation of the new curriculum (Ah- 
mad, 2014). 
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To realize the importance of HOTS, teachers need to teach the skills to students. De- 
signed learning activities should develop students’ HOTS. Some research results indicated that 
it is necessary to alter traditional learning methods to innovative learning methods for learning 
HOTS. Those innovative methods are student-centered learning (Sumarmo & Nishitani, 2010), 
use of constructivism, and provision of opportunities to students for exploring their abilities 
during problem-solving activities (Apino & Retnawati, 2017; Djidu & Jailani, 2016a). Some 
models of learning that belong to innovative learning are problem-based learning (Djidu & 
Jailani, 2016b), project-based learning (Anazifa, 2017), discovery learning (Rochani, 2016), 
and creative problem solving (Apino & Retnawati, 2017). 

Results also indicate that most teachers already know that teaching HOTS to students 
can use various models of learning such as problem-based learning, project-based learning, in- 
quiry learning, and problem-solving. Utilization of HOTS questions with contextual problems 
or PISA problems was also mentioned by the teachers as one of the appropriate strategies to 
train for HOTS. Related to the teachers’ knowledge on teaching HOTS by using various learn- 
ing models, they likely have already been trained about implementing Curriculum 2013. In 
addition, they possibly get information about learning models from teachers’ textbooks or other 
references. However, teachers’ knowledge on various learning models cannot be used as a stan- 
dard for measuring teacher success in teaching HOTS. Teachers also need to know about the 
activities in each particular model of learning so that it can improve their HOTS. It is necessary 
for teachers to pay attention to these activities. 

Based on some previous research, the implementation of mathematics learning on the 
improvement of students’ HOTS can be carried out with some activities, such as involving 
students in non-routine problem-solving activities, providing opportunities to students for con- 
structing their knowledge and improve their ability to analyze, evaluate, and create (Apino 
& Retnawati, 2017), involving students to undergo group discussions, and communicating 
problem-solving results through presentations (Dyjidu & Jailani, 2016b). In other words, build- 
ing HOTS-oriented learning can be conducted through minimalizing teacher domination and 
maximizing the role of students in the learning process. 

Result revealed that teachers have good knowledge on teaching HOTS to students. The 
teachers have mentioned some of the instructional models that contain problem-solving ac- 
tivities. The teachers, however, are confused about explaining the activities that can improve 
students’ HOTS. This shows an inconsistency between the knowledge of teaching HOTS and 
knowledge of activities that can improve HOTS. This also indicates that the pedagogical knowl- 
edge of the teachers on how to learn and improve HOTS is still limited in term of conceptual 
knowledge. 

HOTS-oriented mathematics learning aims at improving students’ HOTS. Measuring 
students’ HOTS in mathematics is important because it helps know whether the purpose is 
achieved or not. Students’ HOTS can be measured through assignments and tests that are con- 
structed based on the aspects and indicators of HOTS. Assignments can be applied by con- 
structing rubrics, but testing can be used with various types of tests, such as multiple-choice 
questions or essay. Both assignment and test have specifications for measuring students’ think- 
ing skills. Multiple choice is more appropriate for measuring analyzing and evaluating skills, 
whereas essay is more appropriate for measuring creating skills. In addition, Watson, Collis, 
Callingha, and Moritz (1995) recommended open-ended questions to measure students’ think- 
ing ability followed by scoring system. The research conducted on 25 mathematics teacher 
candidates in Turkey came up with the findings that teachers still make mistakes in assessing 
students’ thinking ability in making mathematical model of a given problem (Didis, Erbas, Ce- 
tinkaya, Cakiroglu, & Alacaci, 2016). They also showed that there are still many teachers who 
only assess students’ thinking skill based on the final outcome (only providing an assessment: 
true or false, good or bad, appropriate or inappropriate). Meanwhile, only a few students judge 
by observing the process of completion. 
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Compared to the study by Didis et al. (2016), different results were found. Based on the 
analysis of mathematics teacher response data (see Table 5), it can be concluded that teachers 
have a good understanding about assessing students’ thinking ability. It can be seen from teach- 
ers’ responses that measuring HOTS can be carried out by constructing essay with contextual 
problems. Assessment focuses not only on the students’ final answer but also on the process of 
its completion. This result is relevant to that of Altun and Akkaya (2014) that most of teachers 
argue that the cause of students’ low ability in answering questions such as PISA is that the stu- 
dents are unfamiliar with them. Teachers as respondents also provided recommendations that 
evaluation of students’ learning outcome should be carried out by using essays and contextual 
questions. These suggestions show that teachers already know the appropriate types of ques- 
tions to measure HOTS. Some study in some countries (e.g. Altun & Akkaya, 2014; Didis et al., 
2016; Stahnke, Schueler & Roesken- Winter, 2016) revealed that one of the determinant factors 
of student success in improving competence and thinking ability is the teacher competence and 
teachers’ mastery to the learning content. In addition to that, not only mathematics pedagogical 
content knowledge (MPCK), but also increased mathematical content knowledge of teachers 
(MCK) (Blémeke & Delaney, 2012). 

The results of this study indicate that mathematics teachers have not performed well 
in answering HOTS problem. Most teachers do not include clear procedures in answering the 
questions. This is an indication that teachers’ mathematical content knowledge (MCK) is still 
low, especially relating to their ability in solving HOTS-based questions. These results are in 
line with Zulkpli, Mohamed, and Abdullah (2017) who pointed out the low level of thinking 
ability among primary and secondary school teachers across one province in Malaysia. This 
condition will certainly affect students’ learning achievement that is not maximal (Altun & 
Akkaya, 2014; Didis et al., 2016; Stahnke et al., 2016). Besides, these results also indicate 
inconsistency within teachers’ responses in measuring HOTS and answering to HOTS-based 
problems. Although teachers have revealed that assessing HOTS should not ignore the process 
or completion of problem-solving steps, when solving HOTS problem, most of them only write 
the final results and do not include the process of completion. 


Conclusions 


The results of this research indicate that not all teachers understand HOTS well. Teach- 
ers are still unable to distinguish HOTS from ability, skills, learning methods, learning models 
or learning activities. Teachers are already aware of the importance of HOTS for students. 
However, teachers’ knowledge about the importance of HOTS still lacks good knowledge on 
learning and improving HOTS. Although teachers already know that students can be trained 
about HOTS by using some learning models (e.g. problem-based learning, project-based learn- 
ing, inquiry learning, cooperative learning), teachers still confuse activities in particular models 
of learning. The implementation of the Bloom’s Taxonomy has also not been seen in teachers' 
responses for measuring and assessing HOTS. Moreover, the low level of teachers’ knowledge 
on HOTS is also caused by their low ability in solving HOTS problems. The results of this 
study confirm that the low level of pedagogical abilities relating to HOTS is in line with the low 
ability of teachers’ HOTS. 

Although teacher is a key to the implementation of learning process, all educational 
bodies have equal responsibilities for improving education in the country. Results also reveal 
that teachers’ pedagogical knowledge and HOTS are the main concern for educational advance- 
ment. Government and education experts should find solutions to solve this problem. Therefore, 
conducting socialization and teacher training on improving HOTS and its implementation in 
mathematics learning is one of the recommendations that should be applied. In addition, learn- 
ing sources such as books, access to electronic media, or other sources still need further consid- 
eration by the government, schools and education experts. 
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Different types of research can be conducted but they should base on the findings of this 
research. The teachers’ knowledge and skills need to be described more broadly and deeply so 
that it can be easy to identify any side to improve. The strategy to improve the quality of teach- 
ers, especially mathematics teachers, centers on professional and pedagogical competence. 
Similarly, the supporting media needs to develop so that teachers can have access to them in 
order to improve their qualification in managing and implementing innovative teaching and 
learning in their respective schools. 
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